There is a continued desire for size reduction of solid-state camera modules, notably in the consumer mobile market. However, space saving innovations can sometimes have unwanted side effects. One of these side effects is a variation in pixel response across the array, which can arise from the design of the optical components or packaging, or from the design of the actual electronic sensor itself.
One aspect of electronic sensor design that may cause a variation in pixel response is the sharing of amplifying readout circuitry amongst groups of pixels. Typically, readout circuitry may be shared between two rows of pixels. Because the angle of incidence of light impinging on the array varies across the array, the readout circuitry provides a varying “shadowing” effect for different rows.
It is to be understood that the shadowing effect may not correspond to a visible shadow, as the dimensions are too small for this to be the case. The shadowing effect is instead to be thought of more generally as any effect that causes a decrease in the amount of light collected by the pixel under ideal operating conditions. For example, the shadowing effect may be any means that blocks or impedes photons in their path to the pixel, thus reducing the photon count over at least a portion of the pixel, or any means causing stray light gathering, that is, wherein light that should be collected by one pixel is instead collected by another.
This is illustrated in FIG. 1, which shows an array of pixels 10, arranged in rows R1 to Rn and columns C1 to Cm. Successive rows can be labeled as having an “odd” or an “even” parity, in accordance with the parity of the row numbering. Readout circuitry 12 is provided, and as can be seen from the figure is shared between one even and one odd row. The shadowing effect is illustrated by the regions 14-22, which represent an area of each pixel 10 that is relatively poorly exposed with respect to the remainder of the pixel. A lens (not shown) for projecting an image onto the array is cantered at the center point of the array. It can be seen that on an upper portion 24 of the array, the odd rows R1 and R3 are compromised with a shadow effect.
The magnitude of the effect may be greatest at the uppermost edge of the array, where the angle of incident light is the greatest with respect to the normal to the plane of the pixel array. This is represented by the relatively larger area of the region 14 as compared with the region 16. At the center rows R5, R6, the readout circuitry 12 may have a negligible shadow effect 18. On a lower portion 26 of the array, the even rows R8 and R10 are compromised with a shadow effect. Again, the magnitude of the effect is greatest at the uppermost edge of the array, where the angle of incident light is the greatest. This is represented by the relatively larger area of the region 22 as compared with the region 20. Because the shadowing effect changes from odd rows to even rows, it can be thought of as a signal with opposite phases on opposite sides of the pixel array.
Another aspect of electronic sensor design that may cause a variation in pixel response is the crosstalk between components that are introduced by a color filter array used in a color-sensitive image sensor. To produce a color sensitive image sensor, the pixel array is overlaid with a color filter array (CFA), so that each pixel is overlaid with a material with specific transmission characteristics so that the pixel can be thought of as being responsive to a particular color. Usually one color is overlaid per pixel. As a short-hand notation, a “red” pixel is referred to as a pixel that is sensitive to incident light having a frequency in the red part of the visible spectrum of light, with similar notations used for other colors.
The specific frequency responses may vary according to the type of material used, among other factors. There are various well known patterns to use when depositing the color filter material. FIG. 2 illustrates the common Bayer pattern, which comprises red, green, and blue sensitive pixels, labeled R, G, B respectively. Odd rows (R1, R3 etc.) comprise alternate G and R pixels, and so can also be termed as “red” rows, while even rows (R2, R4 etc.) comprise alternate B and G pixels, and so can be termed as “blue” rows.
A source of error in color sensitive image sensors is in the cross-talk of color components, that is, sometimes a photon that passes through a portion of the CFA sensitive to one color actually impacts on the photocollection part of a pixel that is associated with another color. The photoelectric charge generated by the errant photon therefore contributes towards the wrong color channel, providing a source of noise. A uniform crosstalk component may be present, arising from the actual construction of the pixel array.
However, because the CFA has a finite thickness and is overlaid on the pixel array, there may also be a component of crosstalk noise that varies according to the angle of incident light, providing a shadowing effect in a similar fashion to that described above. The shadowing effect is illustrated by the regions 30-48 in FIG. 2. Again, it can be seen that the magnitude of the shadowing effect increases towards the edges of the pixel array, and has a phase change associated with opposite sides of the array. Also, it can be seen that in respect to the red (odd) rows, the shadowing effect comprises the interference of photons that penetrate the blue (even) rows, while for the blue (even) rows, the shadowing effect comprises the interference of photons that penetrate the red (odd) rows.
It may be appreciated that most color-sensitive image sensors may also share readout circuitry as shown in FIG. 1, and so the overall shadowing effect for a given image sensor can arise as a combination of the shared readout circuitry and the crosstalk of color components, and it may be appreciated that there are other aspects of electronic sensor design that contribute to this shadowing effect. The shadowing effect may also vary according to different scene detail and scene luminance to be imaged by the pixel array.
It is also to be appreciated that only the row-to-row variation in pixel response has been discussed and illustrated with respect to FIGS. 1 and 2. The pixel response may also vary in a similar fashion across successive columns of the pixel array. This variation is not illustrated in FIGS. 1 and 2 for the convenience of illustration. The shadowing effect when discussed in general is taken to include both row-to-row and column-to-column variation.
The shadowing effect can be further exacerbated by the design of the optical components or packaging. As camera modules become smaller, the “z-height” (the optical distance between lens and sensor surfaces) can be reduced. However, this widens the angle of incident light incident on the pixel array, and gives a greater variation of angle across the array. This itself may cause a variation in pixel response, as the photoelectric conversion of incident photons can be compromised if the photons are incident at angles that cause the generated electrons to not be collected by the charge collection wells. The increase in variation of angle can also serve to exacerbate the problems of variation of pixel response caused by the design of the actual electronic sensor itself, as described above.
In digital sampling theory, the nyquist frequency is the maximum frequency at which data can be reproduced. It corresponds to half the sampling rate of a digital system. For an image sensor, the nyquist frequency represents the maximum achievable spatial distance of an image that can be reproduced, and is equal to one half the reciprocal of the center-to-center pixel spacing. Because it varies per pixel, row, or column, the abovementioned shadowing effect therefore represents a variation of the pixel response at the nyquist frequency, which we now term for convenience as Nyquist Frequency Variation (NFV) of pixel response.
Currently, there may not be an image sensor that can cancel out NFV, as its effects have not previously been seen as significant when compared with other noise sources. However, with improvements in digital filtering, the effects of NFV are becoming more apparent to Applicant. The effect of NFV may in fact compromise the performance of key signal processing steps in the image reconstruction chain. One such step is noise reduction.
Traditionally, innate noise sources in image sensors are wideband—either Gaussian or impulsive—and techniques have been developed over the years to combat these unwelcome but well-understood defects of image data. However, NFV is narrowband, and its presence serves to alter the statistics of innate noise sources, leading to structured noise artifacts, which are much harder to separate from actual image detail and cancel. In color cameras, a further step (known as “demosaic”) interpolates missing data in the sensor signal to produce a fully-sampled 3-channel color signal. Here NFV appears as a visible “linen” texture in areas of natural low-texture (e.g. blue sky), and breaks up edge detail in the scene representation. Accordingly, there may be a need for image sensors to cancel NFV.